Model-based Clustering With Soft And Probabilistic Constraints

The problem of clustering with constraints has received a lot of attention lately. Many existing algorithms assume the specified constraints are correct and consistent. We take a new approach and model a constraint as a random variable. This enables us to model the uncertainty of constraints in a principled manner. The effect of constraints can be readily propagated to the neighborhood by biasing the search of the optimal parameters in each cluster. This enforces “smooth” cluster labels. The posterior probabilities of these constraint random variables represent the a posteriori enforcement of the corresponding constraints. By combining these probability values with the data likelihood, we arrive at an objective function for parameter estimation. An EM algorithm that maximizes the lower bound of the objective function is derived for efficient parameter estimation, using the variational method. Experimental results demonstrate the usefulness of the proposed algorithm. In particular, our approach can identify the desired clusters when only a small portion of data participate in constraints.